Repository navigation
Conversation
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The automatic outbox loop scans all delivered history on every tick, one failing derivation row stalls derived values for its whole object type, and the bootstrap admin password drops to six digits while the password floor for every account drops to 6.
Corpus store + external extraction agent (stdin/stdout JSON or HTTP), schema compilation, deterministic binder with reference resolution, gated task flow (extract -> review -> confirm; nothing writes before confirm), /admin/documents API surface and the /documents console workspace (source switcher, extractor status, staged runs, extraction graph editor). Platform wiring: kernel, registry, stores, agent broker, builder ops and API factory updated for the new subsystem; web shell, API client and shared components updated accordingly. The semantic kind registry (core/kinds.py) lands here because the whole kernel imports it; its docs and console board follow in the taxonomy commit.
…uring New semiconductor lifecycle showcase (10 objects, 8 links, 16 actions, 4 functions, lifecycle Cedar policies, product/fab/cqe copilots, eval suite, guided demo story) that also serves as the test-suite golden package: it carries the platform coverage the retired package owned — marking (tapeout-version.mask_cost), a derived property (fab_lag_h), all seven effect kinds (webhook/set-link/archive-object included), equipment reassignment and an intentional RSI demo gap (field-failure lot_id left unmodeled on purpose). product-manufacturing is removed together with its docs and tools; the document-intelligence FMEA case moves to the module FMEA agent (tools/doc_agent_fmea_module.py) dispatched by the shared demo router. All domain-touching tests are re-targeted at chip-lifecycle seed data with their platform assertions preserved; the two superseded contract files are dropped (covered by test_chip_lifecycle.py).
…integration - Kind taxonomy: core/kinds.py docs counterpart (00-kind-taxonomy zh/en), frozen-table contract test, console arch board reading the registry via kindVisuals; board caption counts corrected (9 platform resources of 11 kinds). - Python 3.12+: ontogeny requires-python/classifiers raised, CI matrix pins bumped, py312 per-file ruff target with 58 mechanical modernizations (datetime.UTC alias, TimeoutError), README floor statement and test badges updated. - Selective CI: ontogeny registered in scripts/ci.py (module suite + 3.12-3.14 full matrix), ontogeny-ci.yml converted to the reusable ontogeny.yml workflow wired into ci.yml preflight/result, selection tests updated; docs/quality/ci.md documents the Ontogeny suite. - Docs: reference manual gains the documents API group and /documents route (104 operations / 92 paths, counted from live OpenAPI), the UI tour gains the document-intelligence page (zh/en, real screenshot), quickstart/UI-tour sample-domain references now chip-lifecycle, the obsolete merge-plan scratch doc is removed, web e2e and bench target the chip domain.
…ence + chip-lifecycle Merge the mainline's 55-commit hardening wave (sandbox bounds, approval policies against unmasked state, idempotency scoping, cross-domain table guards, typed JSON columns, watermark boundaries, staged builder saves, PostgreSQL-gated concurrency suites, Python 3.12 floor) with this branch's document intelligence subsystem and chip-lifecycle golden domain. Conflict resolution kept both sides' semantics: - service.py: mainline _install_snapshot (cache snapshot + rollback) with the layout-store rebind re-applied inside it (switching domains must rebind the canvas layout -- unfixed on the mainline) - api_factory: mainline try/lifespan structure + the document-extract skip in the demo populate loop - db.py: both sides' additive column migrations (documents tables + outbox cursor receipts) - builder_ops: mainline staged/journaled save rewritten onto the core/kinds.py KIND_DIRS registry (module-scope import -- the mainline's new _materialize also consumes it) - AgentBroker lost its task-handler surface on the mainline: document extraction now runs the doc-agent protocol directly (no session per extraction payload); the broker-path test now pins the direct run - tests: the mainline's new suites (effect ordering, agent atomicity, idempotency scope, marked filters, typed storage, PG gates, ...) are re-targeted at chip-lifecycle seed data with their platform assertions intact; PLANNER-era helper aliases kept for cross-suite imports - docs: mainline monorepo README + this branch's badges/wording; the mainline's hardened quickstart (deploy/credentials/auth defaults) with the minimal-package section retained; validator list takes the mainline's markings-on-PK additions - CI: the reusable ontogeny.yml keeps the mainline's frontend lint and PostgreSQL concurrency job, parameterized through the selective-CI inputs; scripts/tests floor assertions follow the parameterized wiring Verified on Python 3.12: 898 backend tests passed (31 by-design skips: live-LLM/PG-gated), web 278 + build, ruff clean, CI selection self-tests green, demo smoke (document extraction 7/7 + tapeout chain) green.
The apache/skywalking-eyes preflight gate scans every tracked file outside the licenserc ignore list (.md/.sql/.cedar/.json/.github etc.), and the files introduced with the document-intelligence and chip-lifecycle work shipped without the header -- which failed the preflight license step and skipped every selected module job on PR #90. Headers added to: chip-lifecycle package YAML (objects/links/actions/ functions/policies/projections/stores/evals/agents/layout/ontology), the supplier-risk test fixture package, the deterministic doc agents (demo/reference/openai/module-FMEA) and the shared document test fixtures. SQL and Cedar sources stay headerless: the ignore list covers them and '#' breaks both syntaxes. Local reproduction of the licenserc scan (same ignore rules) now reports zero missing headers; full suite re-verified 898 passed.
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: Document extraction drops non-snake_case properties, rewrites numeric-looking strings, breaks a store when a removed file returns, re-raises rejected candidates and mixes attempt ids with run ids; the three earlier comments still apply at this head.
The earlier review comments below still apply at 0e0b6ab; this head does not fix them.
- Important: this anti-join starts at the first outbox row and nothing prunes
ontogeny_outboxorontogeny_outbox_receipt, so each 1-second tick runs four scans whose cost grows with all delivered history (every sync adds one row per source row). A local SQLite run with 1M delivered events spends about 1 s per idle tick. Keep a per-consumer floor below which every ID is acknowledged and addOutboxRow.id > floorto the query. (#391 (comment)) - Important: a row whose function raises fails the whole
map_rowsbatch, and its pending entry now persists, so every later drain includes it and no row of that type gets new derived values while it stays pending. Retry a failed batch row by row and leave only the rows that still fail pending. (#391 (comment)) - Important: the generated admin password has 10^6 values, the username defaults to
admin, and/auth/loginhas no attempt limit, whileMIN_PASSWORD = 6lowers the floor for every account. Generate a high-entropy bootstrap secret such assecrets.token_urlsafe(24)and keepMIN_PASSWORD = 8. (#391 (comment))
|
Follow-up at head Remaining confirmed boundaries:
Validation: 156 targeted backend tests passed and 4 PostgreSQL-dependent tests were skipped in that run; a separate run against a real PostgreSQL 16 instance passed 12 outbox/lifespan tests, and the frontend access tests passed (19 tests). The combined backend run emitted three aiosqlite event-loop-closed warnings. No production deployment or old-data migration was validated. |
|
Independent follow-up score: 9.2/10 for the reviewed fixes in The document ID collision, historical data and deployment boundaries remain. The exact-head CI is still running, and #380 remains open. |
|
Follow-up: the cross-store document ID collision noted above is fixed in Rollback is not seamless: back up the database and corpus together before upgrading, and restore the matching backup if downgrading; that loses changes since the backup. If both legacy and scoped records already exist for the same document, scanning stops before altering references and requires deliberate reconciliation. This change does not guess or rewrite historical attempt/run associations. CI for this exact head is pending. |
|
Follow-up at This does not reconstruct or migrate the ambiguous historical attempt/run associations described above: the original rows do not contain enough information to identify their provenance reliably. Existing limitations on rollback, single-process dispatch, and the open #380 dependency still apply. The 77 evolve and query-service tests passed after integrating the concurrent branch changes; CI for this new head is pending. |
Independent review follow-upReviewed head
No further actionable findings were identified in those scopes. These scores are not a review of all files in the PR. The reviewers performed static inspection and did not independently run tests. The implementation team reports 77 related evolve/query tests passing, plus targeted PostgreSQL, SQLite and document-ingestion tests. Limits: stop the old worker before upgrading and run only one service process per database. Historical delivery gaps require audit and scoped replay, which can duplicate delivery; old audit records lack schema versions and cannot safely be backfilled automatically. Current-head CI is still running, and #380 remains an upstream dependency. |
|
Independent verification at Reproduction (Linux)Environment: Ubuntu 22.04 x86_64, Python 3.12.13 (uv-managed), SQLite/aiosqlite, single process. Ran the regression suites for the eight reviewed fixes plus the historical-replay fix:
Result: 104 passed, 2 skipped (the two Not covered here: the The same nine files were also run on Windows 10 (zh-CN, Python 3.12) to check platform assumptions. That surfaced the findings below. One further Windows failure, Finding 1 —
|
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The earlier outbox, derivation, bootstrap-password and ingestion fixes hold at this head, but the demo still accepts the X-Ontogeny-Principal admin header the docs now call disabled, a superseded document task can be re-confirmed over newer data, reviewer overrides bypass type checks, agent-path rule and policy refusals no longer reach the revision audit, and --reseed leaves a credential file that stops the next start. The Minor items are smaller lifecycle, message and docs issues.
|
|
||
| > [!NOTE] | ||
| > The API is session/bearer authenticated. The `X-Ontogeny-Principal` dev header is **enabled by default** for evaluation (`ONTOGENY_DEV_AUTH=0` disables it) — turn it off in production, where the agent surfaces (REST + `/mcp`) then require a signed-in driver and every session records WHO drove it. | ||
| > The API is session/bearer authenticated. The `X-Ontogeny-Principal` dev header is disabled by default; set `ONTOGENY_DEV_AUTH=1` only for development or evaluation. With it disabled, the agent surfaces (REST + `/mcp`) require a signed-in driver and every session records WHO drove it. |
There was a problem hiding this comment.
Important: ontogeny serve --demo, which the Docker CMD and the quickstart both run, still sets ONTOGENY_DEV_AUTH=1 through DemoPaths.as_env, so any client that sends X-Ontogeny-Principal: {"id":"x","is_admin":true} passes require_admin while this note says the header is disabled by default. Stop --demo from enabling dev auth unless the operator sets ONTOGENY_DEV_AUTH=1.
| task = await session.get(DocumentTaskRow, task_id) | ||
| if task is None: | ||
| raise NotFoundError(f"no task {task_id}") | ||
| if task.status == "confirmed": |
There was a problem hiding this comment.
Important: confirm refuses only confirmed tasks, so a superseded revision can be confirmed again and rewrites its older graph over the newer confirmed one, leaving both tasks confirmed; TasksTab.tsx shows the Confirm button for it. Refuse confirm, save_graph and run unless the task is still open for review.
| try: | ||
| props[prop] = coerce_value(ptype.ptype(), value, prop=prop) | ||
| except Exception: | ||
| props[prop] = value |
There was a problem hiding this comment.
Important: when coerce_value rejects a reviewer override, this fallback writes the raw value, so {"risk_level": "CATASTROPHIC"} is stored on an enum[LOW, MEDIUM, HIGH] property and effective_date: "not a date" fails later as a 500. Raise a DocumentError on a coercion failure instead of storing the raw value.
| params, | ||
| target_id, | ||
| expected_revision=args.get("expected_revision"), | ||
| commit_rejections=False, |
There was a problem hiding this comment.
Important: with commit_rejections=False inside begin_nested(), a rule-rejected or policy-denied agent write rolls back its rejected_rule/denied_policy revision with the savepoint, here and in the approval path of decide, so agent refusals no longer reach the audit stream that REST refusals still write. Record the refusal revision after the savepoint rollback.
| try: | ||
| fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) | ||
| except FileExistsError as exc: | ||
| raise StoreError( |
There was a problem hiding this comment.
Important: ontogeny serve --demo --copy-package --reseed deletes ontogeny.db but keeps admin-bootstrap-password, so the next start finds no admin, raises here and exits with Application startup failed. Remove the stale credential file in prepare_demo when --reseed deletes the database.
| wanted_ids = set(doc_ids or []) | ||
| async with self.sc.sessionmaker() as session: | ||
| docs = (await session.execute(select(DocumentRow).where(DocumentRow.store == store_name))).scalars().all() | ||
| selected = [d for d in docs if d.id in wanted_ids or Path(d.path).resolve() in paths] |
There was a problem hiding this comment.
Minor: this selects removed rows whose path matches, so uploading a revised file under the same name returns a second task bound to the removed document id that reads the new bytes. Select only active documents.
| issues = [{"message": "issues must be an array"}] if issues is not None else [] | ||
| status = chunk.get("status", "final") | ||
| if status not in _VALID_STATUS: | ||
| status = "failed" |
There was a problem hiding this comment.
Minor: status is reassigned before the issue is built, so the message always reads invalid status 'failed' and drops the value the agent sent. Build the message from the original value first.
| raise SkillError(f"skill examples must be a list of at most {MAX_EXAMPLES} items") | ||
| if len(json.dumps(examples, ensure_ascii=False, default=str)) > MAX_EXAMPLE_CHARS: | ||
| raise SkillError("skill examples are too large") | ||
| language = str(skill.get("language") or "en") |
There was a problem hiding this comment.
Minor: the snapshot always stores language, defaulting to en, so skill.get("language") or cfg.language in tasks.py never falls back to the store's documents.language (default zh). Leave language unset when the caller gives none.
| disabled={createDefault.isPending} | ||
| data-testid="documents-create-default" | ||
| > | ||
| {createDefault.isPending ? t('documents.default.creating') : t('documents.default.action')} |
There was a problem hiding this comment.
Minor: documents.default.action and documents.default.creating exist in neither en.ts nor zh.ts, so the create-default button shows the raw key. Add both keys to the two dictionaries.
| ``` | ||
|
|
||
| `tools/doc_agent_reference.py` is a deterministic reference agent that makes the | ||
| Product Manufacturing demo runnable without an LLM; `tools/doc_agent_openai.py` shows the |
There was a problem hiding this comment.
Minor: this names the Product Manufacturing demo, which this PR deletes in favour of chip-lifecycle, and line 23 reads Baselineal guardrails. Name the chip-lifecycle demo and fix the heading.
Before → After
Before: The pending Ontogeny proposal needs further review fixes for runtime consistency, lifecycle handling and compatibility.
After: Apply the follow-up fixes and regression coverage from the existing fork branch: object/version and idempotency consistency, transaction/outbox lifecycle, policy and identity handling, typed synchronization, and console/deployment behavior.
Dependency and scope
Depends on 🔗 AI #380. This stacked PR targets
apache:ontogeny, notmain, and contains 55 commits beyond that proposal's current head (155 changed files). It preserves the existinghugegraph:ontogenysource branch and avoids duplicating the original platform introduction as a second PR intomain. Recheck the base and remaining diff after #380 merges; do not assume squash or branch deletion preserves this comparison.Validation and upgrade boundaries
The author reports backend/frontend CI, PostgreSQL concurrency checks and isolated HugeGraph projection smoke tests. These results have not been independently reproduced by this submission check. Existing databases may require explicit migrations/backfills, and automatic outbox processing can deliver queued external events. See the source PR's detailed compatibility and deployment notes before adoption. This submission is not release, live-cluster or UI acceptance.
Source and detailed evidence: 🔗 hugegraph/hugegraph-ai #90
Visual overview
The diagram illustrates the shared architecture and the proposed stacked relationship between #391 and #380; it does not indicate that either PR has merged or that release acceptance has passed.